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Abstract.  In  this  paper  we  present  a  hybrid  systems  approach  to  freeway 
traffic  control.  We  introduce  a  model  of  traffic  which  includes  both  vehicles 
and  voids,  where  a  void  is  a  unit  of  space  greater  than  the  normal  following 
distance  between  vehicles.  The  introduction  of  voids  allows  us  to  successfully 
model  traffic  flow  via  a  wave  model.  We  discuss  the  fundamentals  of  the 
Multiple  Agent  Hybrid  Control  Architecture  which  was  used  to  implement 
our  control  of  freeway  traffic  and  the  results  of  a  simulation  of  our  control. 
The  main  conclusion  of  our  simulation  is  that  one  can  significantly  increase 
traffic  throughput  by  controlling  only  5-9%  of  the  vehicles. 


1  Introduction 

This  paper  presents  an  architecture  for  the  real-time  feedback  control  of  hybrid  sys¬ 
tems  through  a  communication  network  composed  of  multiple  decision  makers  herein 
refereed  to  as  agents.  The  paper  discusses  some  recent  results  from  the  theory  of  Hy¬ 
brid  systems,  developed  by  the  authors,  related  to  the  behavior  of  the  architecture 
and  illustrates  them  with  a  simple  highway  control  system.  This  application  was 
selected  because  it  shares  many  elements  of  commonality  with  many  other  areas 
of  application  such  as  distributed  control,  sensor  fusion,  manufacturing  shop-floor 
control  and  hybrid  routing  in  communication  networks;  and  yet  it  requires  relatively 
modest  modeling  efforts. 

Our  architecture  implements  formal  real-time  intelligent  controllers  with  learn¬ 
ing  capabilities.  This  architecture,  termed  the  Multiple-Agent  Hybrid  Control  Ar¬ 
chitecture  (MAHCA),  [21,  22],  provides  a  knowledge-based,  formal  implementation 

*  Research  supported  by  Dept,  of  Commerce  Agreement  70-NANB5H1164. 

**  Research  supported  by  ARO  under  the  MURI  program  “Integrated  Approach  to  Intelli¬ 
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**  Research  supported  by  Dept,  of  Commerce  Agreement  70-NANB5H1164. 
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framework  for  deducing  on-line  feedback  control  and  reactive  strategies  for  processes 
involving  multiple  agents.  The  architecture  includes  capabilities  for  structural  adap¬ 
tation  as  a  function  of  predictable  and  unpredictable  events  in  the  processes  imder 
control.  This  characteristic  is  necessary  for  satisfying  performance  requirements  in 
the  unavoidable  presence  of  sensory  and  knowledge  imcertainty. 

We  will  illustrate  the  functional  and  operational  characteristics  of  MAHCA  in 
terms  of  a  two-agent  controller  version  whose  goals  are  to  maximize  traffic  through¬ 
put  in  a  freeway  network  and  to  provide  a  dynamic  route  planning  for  selected  cars 
through  the  network  with  minimum  average  time.  The  highway  example  was  chosen 
because  it  explicitly  exhibits  some  the  basic  properties  of  hybrid  systems  in  a  clear 
fashion.  In  additinn  we  were  able  to  demonstrate  a  property  of  these  systems  which 
is  unique.  That  is,  one  can  achieve  a  significant  increase  in  throughput  by  control¬ 
ling  a  relatively  small  number  of  vehicles.  This  result  is  a  consequence  of  the  implicit 
inter-vehicle  constraints  that  limit  the  degrees  of  freedom  for  control  in  the  freeway 
to  a  small  percentage  of  the  total  vehicle  by  vehicle  aggregated  degrees  of  freedom. 
This  implies  that  a  platooning  effect  is  achieved  spontaneously  by  controlling  the 
velocity  of  about  9-14%  of  the  cars  in  the  freeway  network. 

The  organization  of  this  paper  is  as  follows.  In  section  2,  we  shall  provide  an 
overview  of  our  freeway  network  model.  In  section  3,  we  shall  descibe  the  generic 
variational  model  of  MAHCA  agent.  In  section  4,  we  shall  give  an  overview  of  the 
architecture  of  MAHCA  agent  and  how  it  carries  out  its  computations.  Finally  in 
section  5,  we  will  discuss  the  results  of  a  simulation  of  our  highway  control.  In 
particular,  at  the  end  of  the  paper,  we  will  provide  a  series  of  charts  which  give  the 
results  of  controlling  5%,  8%,  and  9%  of  the  vehicles. 


2  Freeway  Network  Model  and  Control 

The  freeway  network  model  we  will  use  in  our  illustration  of  MAHCA  is  composed 
of  two  elements  the  Network  Geometry  and  the  Network  Dynamics.  We  discus  these 
elements  in  the  next  two  subsections. 

The  substance  in  a  traffic  flow  is,  of  course,  vehicles.  However,  in  addition  to 
modeling  the  flow  of  vehicles,  the  model  also  includes  the  flow  of  voids,  where  a 
void  is  a  unit  of  space  greater  than  the  normal  following  distance  between  vehicles. 
As  vehicles  (referred  to  generically  as  “cars”  in  what  follows)  travel  from  a  source 
(a:  =  0)  to  the  destination  (x  =  L),  the  voids  travel  in  the  reverse  direction,  i.e. 
starting  at  x  =  L. 

The  introduction  of  voids  eliminates  a  major  problem  with  wave  models.  That  is, 
such  models  are  only  valid  for  high  uniform  densities.  Modeling  the  voids  and  their 
interaction  with  cars  permits  the  generation  of  more  realistic  density  distributions. 
When  the  voids  are  also  modeled,  the  sparse  density  of  vehicles  become  high  densities 
of  voids,  allowing  for  model  validity  at  low  vehicle  densities.  This  representation  is 
patterned  after  the  model  of  propagation  of  electrons  and  holes  in  semiconductor 
junctions. 

For  incorporating  driver  behavior  and  policies  we  use  a  Toda  lattice  representa¬ 
tion  vehicle  interaction  for  each  policy  and  generate  the  composite  wave  model  via 
a  formal  aggregation  procedure  similar  to  the  one  proposed  in  [44]. 


A  freeway  segment  is  treated  a  single  pipe  with  a  capacity  density  corresponding 
to  an  average  lane  car  velocity  of  50  miles/hr  for  the  number  of  lanes  considered. 
Individual  lanes  are  not  modeled.  Accidents  are  represented  as  reductions  in  the 
capacity  of  the  segment. 


2.1  Network  Geometry 

The  network  geometry  is  represented  by  a  segment  directed  graph  (SDG).  An  SDG 
is  a  structure  composed  of  two  types  of  sets:  an  edge  set  (E)  and  a  node  set  (N).  The 
set  of  edges  (E),  represent  unidirectional  freeways.  Each  edge  (freeway)  is  modeled 
as  a  composite  of  one  or  more  segments.  The  set  of  nodes  (N)  is  composed  of  three, 
not  necessarily  disjoint,  subsets:  sources  (S),  interior  nodes  (I)  and  destination  nodes 
(D).  A  source  node  represents  a  point  at  which  cars  enter  the  network,  a  destination 
node  is  a  point  at  which  cars  leave  the  network  and  an  interior  node  a  point  at 
which  cars  flow  from  one  or  more  freeway  segments  to  other  freeway  segments  These 
concepts  are  illustrated  in  Figure  1. 

Source  Node  Edge  (Freeway)  DesitinationNode 


Freeway  Segments 


Fig.  1.  Geometry  of  the  Model 


FVom  the  geometric  point  of  view,  an  edge  is  a  connected  straight  line  with  a  di¬ 
rection,  indicated  by  an  arrow  at  its  destination  point,  containing  two  or  more  nodes. 
An  edge  is  composed  of  one  or  more  segments  where  a  segment  is  a  line  in  between 
two  nodes.  A  two-way  freeway  is  represented  by  two  edges  with  opposite  directions. 
Thus,  independently  of  the  actual  map  characteristics  of  a  freeway  segment,  in  our 
model  a  freeway  segment  is  represented  by  a  straight  line  which  we  assume  has  the 
same  length  as  the  freeway  segment  it  represents.  This  simplifying  assumption  does 


not  cause  inaccuracies  because  we  model  the  constraints  due  to  the  geography  and 
other  constraints  on  traffic  flow  with  a  capacity  function  that  we  will  describe  in  the 
next  subsection. 

Figure  2  shows  the  geometric  model  of  the  freeway  system  of  the  greater  Houston 
area.  This  network  was  used  to  exercise  our  2-agent  implementation  of  MAHCA. 
Some  of  the  simulation  results  will  be  presented  in  section  5. 


Fig.  2.  Houston  Highway  Network 


2.2  Freeway  Network  Dynamics 

For  the  purpose  of  representing  traffic  flow  dynamics,  we  view  the  freeway  segments 
as  pipes  with  variable  cross-section  carrying  a  composite  fluid  of  two  types  of  inter¬ 
acting  particles,  “car”  and  “void”  particles.  The  void  particles  represent  statistical 


averages  of  inter-car  distances.  In  each  segment,  cars  flow  in  the  direction  of  the 
segment  edge  while  voids  flows  in  the  opposite  direction. 

The  central  principle  behind  our  dynamics  model  of  freeway  networks  is  that  the 
number  of  cars  and  voids  flowing  through  a  freeway  segment  are  conserved  quantities. 
This  principle  is  an  extension  of  the  one  proposed  [16]  and  [24]  and  is  now  widely 
used  in  analysis  of  freeway  control  [25]. 

Rather  than  introduce  the  principle  in  its  more  general  form,  we  will  motivate 
it  informally  from  a  simple  particle  model  of  single  lane  freeway  segments.  This 
exercise  is  important  because  many  aspects  of  freeway  dynamics  [24],  such  as  driver 
behavior,  are  easily  stated  in  terms  of  particle  models. 

The  general  idea  behind  particle  models  in  a  single  lane  freeway  segment  is  the 
following.  We  set  up  a  coordinate  system  with  origin  at  the  beginning  node  of  the 
segment  (we  assume  that  a  freeway  segment  inherits  the  direction  of  its  edge).  We 
AjaaiiTTiP!  that  there  are  on  the  average  M  cars  and  M  —  1  voids  in  the  segment  and 
that,  the  speed  at  which  the  n-th  car  and  the  n-th  void  move,  at  time  t  are  functions 
of  the  positions  of  the  (n  -  l)-th  and  (n  +  l)-th  cars  and  the  (n  —  l)-th  void  in  the 
lane.  In  symbols, 

in(^  "b  — /(®n— 1(^)>  ®n— 1(^))  ®f»+l(^))  (1) 

where  respectively  the  position  of  the  A:-th  car  and  1-th  void  in 

the  segment  at  time  t  and  /  and  g  are  functions  that  capture  driver  behavior,  car 
characteristics  and  geographic  and  environmental  constraints.  For  the  n-th  car  and 
n-th  void  (n  =  1, M )  in  the  segment,  the  functions  /  and  g  map  the  positions  of 
neighboring  cars  and  voids  at  time  t  to  the  speeds  of  the  n-th  car  and  n-th  void  at 
time  t  +  6  where  is  a  positive  real  number  modeling  average  driver  reaction  time. 

For  the  purposes  of  characterizing  highway  segment  velocity  control  and  car 
routing  through  the  network,  the  model  above  is  not  satisfactory.  We  need  a  model 
that  gives  a  global  view  of  the  dynamics.  With  this  goal  in  mind  we  have  transformed 
the  particle  model  above  into  a  wave  model  [25].  In  a  wave  model,  the  cars,  the  voids 
and  their  interactions,  in  each  freeway  segment,  are  expressed  by  a  set  of  coupled 
partial  differential  equations  expressing  conservation  of  mass  (of  cars  and  voids)  and 
velocity  of  cars  and  voids.  These  equations  are  given  in  (3)  -  (5)  and  (6)  and  (7) 
below. 


^  +  ^  +  hiC‘-p^)p'’-k2(C”-p”)  =  0  (3) 

with 

Q’  =  W)-d% 


%  +  ^  +  -  /)  =  0 
with 


(4) 


In  (3)  and  (4),  t  is  time,  x  is  the  freeway  space  variable,  fp,  for  j  =  c  or  v, 
are  the  densities  of  the  car  and  void  particles  (cars/mile,  and  voids  /mile),  for 
j  =  c  or  Vy  are  the  flows  of  cars  and  voids  (cars  /sec  and  voids/sec)  respectively. 
The  capacity  of  the  freeway  segment  at  each  point  is  given  by  the  functions  , 
for  j  =  c  or  Vy  in  cars/sec  and  voids/sec  respectively.  The  coefficients  ki  and  k2 
are  empirical  coefficients  expressing  the  transference  from  voids  to  cars  and  cars  to 
voids  respectively.  Finally,  d  and  di  are  aggregated  empirical  diffusion  coefficients 
expressing  driver  behaviOT  dependencies  on  the  neighboring  cars  and  voids.  In  (3) 
and  (4),  the  functions  Qj,  j  =  c  for  cars  or  j  =  u  for  voids,  are  called  the  free  flows 
(for  cars  and  voids  respectively  and  measuring  the  number  of  cars  and  voids  per 
second),  and  are  given  by 


with  J  =  c  or  J  =  u 


(5) 


where  and  are  empirical  coefficients.  In  particular  for,  Greenber’s  law  of  traffic 
flow  for  freeway  segments  under  heavy  load  [24],  (5)  is  the  Fade  approximant  to  the 
natural  logarithmic  function. 

Equations  (6)  to  (8)  express  the  wave  velocity  dynamics  for  j  =  cor  u,  of  cars 
and  voids  along  a  freeway  segment.  In  (6),  kz  and  represent  the  control  flelds 
in  each  segment.  is  the  recommended  wave  velocity  and  ks  is  the  percentage  of 
cars  that  are  controlled.  Since  the  dynamics  of  an  individual  car  is  constrained  by 
its  neighbors  the  controller  does  not  have  to  control  all  the  cars  on  the  segment  to 
achieve  the  desired  goal  which  is  to  maximize  throughput  for  the  freeway  network. 


+  (6) 

+  (7) 

The  equations  in  (8),  express  the  boundary  conditions  for  each  freeway  segment. 
The  right-hand  side  of  the  first  equation  expresses  the  gradient  of  the  density  of 
cars  and  voids  being  pumped  into  the  segment.  The  right-hand  side  of  the  second 
equation  in  (8)  expresses  the  density  of  particles  leaving  the  segment. 

0)  =  P^{t)  j  =  VOTC  (8) 

with 


.dv‘ 

dv'’ 


The  exchange  between  voids  and  cars  is  given  by 


dp<‘  V  dQf, 

dt  dt  dx  dx 


(9) 


This  equation  forms  the  rule  which  guarantees  the  conservation  of  the  total  number 
of  “units”  (cars  and  voids)  in  the  freeway  system. 

The  model  outlined  above,  characterizes  the  dynamics  of  freeway  segments  and 
hence  the  freeway  network.  This  model  constitutes  the  specific  (problem  dependent) 


equational  knowledge  required  to  control  the  freeway  network  using  MAHCA.  The 
next  subsection  formulates  the  highway  velocity  control  problem  as  a  multiple  agent, 
knowledge-based  control  problem. 

2.3  Control  Problem 

The  overall  goal  of  the  control  system  is  to  maximize  throughput  for  the  freeway 
network  by  controlling  the  velocity  and  the  percentage  of  controlled  cars  fcs, 
referred  to  as  the  control  actions,  in  each  segment.  We  associate  with  each  segment  i 
a  control  agent  A*  which  generates  the  control  actions  for  the  corresponding  segment 
as  a  function  of  sensory  data,  goal  data,  current  status  data  and  information  from 
the  other  agents  via  a  communication  network  herein  referred  to  as  the  Control 
Network  A  formal  model  of  the  control  network  and  its  dynamics  is  given  in  section 
3. 

3  Control  Network 

3.1  A  MAHCA  Agent’s  Model 

In  general,  a  hybrid  system  has  a  hybrid  state,  the  simultaneous  dynamical  state 
of  all  plants  and  digital  control  devices.  Properly  construed,  the  hybrid  states  will 
form  a  differentiable  manifold  which  we  call  the  comer  manifold  of  the  system.  To 
incorporate  the  digital  states  as  certain  coordinates  of  points  of  the  carrier  manifold, 
we  “continualize”  the  digital  states.  That  is,  we  view  the  digital  states  as  finite,  real¬ 
valued,  piecewise-constant  functions  of  continuous  time  and  then  we  take  smooth 
approximations  to  them.  This  also  allows  us  to  consider  logical  and  differential  or 
variational  constraints  on  the  same  footing,  each  restricting  the  points  allowed  on 
the  carrier  manifold.  In  fact,  all  logical  or  discontinuous  features  can  be  continualized 
without  practical  side-effects.  This  is  physically  correct  since  for  any  semiconductor 
chip  used  in  an  analog  device,  the  zeros  and  ones  are  really  just  abbreviations  for 
sensor  readings  of  the  continuous  state  of  the  chip.  Every  constraint  of  the  system, 
digital  or  continuous,  is  incorporated  into  the  definition  of  what  points  are  on  the 
carrier  manifold.  Lagrange  constraints  are  regarded  as  part  of  the  definition  of  the 
manifold  as  well,  being  restrictions  on  what  points  are  on  the  manifold. 

More  specifically,  let  Ai,  f  =  1, . . . ,  N{t)  denote  the  agents  active  at  the  current 
time  t  .  In  our  model,  t  takes  values  on  the  real  line.  At  each  time  i,  the  status  of 
each  agent  in  the  network  is  given  by  a  point  in  a  locally  differentiable  manifold 
M  [24].  The  Behavior  function  Bi  of  an  active  agent  A*  is  given  by  a  continuous 
function, 

BiiMxT^R^  (10) 

where  T  is  the  real  line  (time  space)  and  ii"*"  is  the  positive  real  line.  M  is  contained 
in  the  Cartesian  Product 

MCGxSxXxA  (11) 

where  G  is  the  spaces  of  goals,  S  is  the  space  of  sensory  data,  X  is  the  space 
of  controller  states  and  A  is  the  space  of  control  actions.  In  the  freeway  network, 
X  is  the  space  of  current  distributions  of  car  and  void  densities  and  velocities  in 


its  segments,  5  is  the  space  of  measurements  of  car  and  void  densities  along  the 
segment  (one  sensor  each  third  of  a  mile),  A  is  the  space  spanned  by  the  velocity 
and  percentage  fields,  and  G  is  the  set  of  car  and  void  densities  that  maximizes 
throughput. 

From  an  agent’s  point  of  view,  the  dynamics  of  the  control  network  is  charac¬ 
terized  by  certain  trajectories  on  the  manifold  M.  These  trajectories  characterize 
the  flow  of  information  through  the  network  and  its  status.  Specifically,  we  need  to 
define  two  items: 

(i)  a  set  of  generators  for  the  behavior  functions  at  time  i, 

{BiM:i  =  h...N,peM}  (12) 

and 

(ii)  the  control  actions  issued  by  the  agents. 

We  will  see  shortly  that  these  actions  are  implemented  as  infinitesimal  transforma¬ 
tions  defined  in  M.  The  general  structure  of  the  behavior  function  in  (12)  for  an 
agent  ^4^  at  time  t  is  given  in  (13)  below: 

(13) 

where  Fi  is  a  smooth  function,  B  is  the  vector  of  behavior  functions,  C7“  is  the 
behavior  modification  function  for  the  i-th  agent,  and  ai  is  the  command  action 
issued  by  the  z-th  agent.  We  will  devote  the  rest  of  this  subsection  to  characterizing 
this  model. 

We  start  with  a  discussion  of  the  main  characteristics  of  the  manifold  M.  In 
general  a  manifold  M  is  a  topological  space  (with  topology  0)  composed  of  three 
items: 

(a)  A  set  of  points  of  the  form  of  (11). 

(b)  A  countable  family  of  open  subsets  of  M,  Ui  such  that 

\JUi  =  M. 

i 

(c)  A  family  of  smooth  homeomorphisms,  {<f>i  :  <t>i:Ui-¥  VJ},  where  for  each  7,  Vj 
is  an  open  set  in  ii*.  The  sets  Ui  are  referred  to  in  the  literature  as  coordinate 
neighborhoods  or  charts.  For  each  chart  Uj  the  corresponding  function  is 
referred  to  as  its  coordinate  chart. 

The  coordinate  chart  functions  satisfy  the  following  additional  condition: 

Given  any  charts  Ui  and  Uj  such  that  Ui  H  Uj  ^  0,  the  function  ^i  o  (j>J^  : 
<l>j{Ui  n  Uj)  ->>  (t>i{Ui  n  Uj)  is  smooth. 

In  the  literature,  one  usually  finds  an  additional  property,  which  is  the  Hausdorff 
property  in  the  definition  of  manifolds  [25].  Since  this  property  does  not  hold  in  our 
application  we  will  not  discuss  it. 

Now  we  proceed  to  customize  the  generic  definition  of  the  manifold  to  our  appli¬ 
cation.  We  start  with  the  topology  0  associated  with  M.  We  note  that  the  points 


of  M  have  a  definite  structure,  see  (11),  whose  structure  is  characterized  by  the 
interval  of  values  in  the  space  G  of  goals,  the  space  S  of  sensory  data,  the  space  X  of 
controller  states  and  the  space  A  of  control  actions.  The  number  of  these  parameters 
equals  k.  The  knowledge  about  these  parameters  is  incorporated  into  the  model  by 
defining  a  finite  topology  O  on  i?*  [5]. 

The  open  sets  in  H  are  constructed  firom  the  clauses  encoding  what  we  know 
about  the  parameters.  The  topology  0  of  M  is  defined  in  terms  of  /?  as  follows.  For 
each  open  set  W  ia  Q  such  that  W  CVj  C  i?*,  we  require  that  the  set 
be  in  0.  The  sets  constructed  in  this  way  form  a  basis  for  0  so  that  a  set  17  CM 
is  open  if  and  only  if  for  each  p  €  17,  there  is  j  and  an  open  set  W  €  12  such  that 
W  C  Vj  and  pe<j>J^iW). 

To  characterize  the  aictions  conunanded  by  a  MAHCA  agent  we  need  to  introduce 
the  concept  of  derivations  on  M.  Let  Fp  be  the  space  of  real  valued  smooth  functions 
/  defined  in  a  neighborhood  a  point  p  in  M.  Let  /  and  g  be  functions  in  Fp.  A 
derivation  v  of  Fp  is  a  map 

v:Fp-¥Fp 

that  satisfies  the  following  two  properties. 

«(/  +  fl)  (P)  =  {«(/)  +  v(p))  (P)  (Linearity)  (14) 

vif  •  p)(p)  =  (v(/)  •  P  +  /  •  Hff))(p)  (  Leibniz  Rule)  (15) 

Derivations  define  vector  fields  on  M  and  a  class  of  associated  curves  called 

integral  curves  [26].  Suppose  that  C  is  a  smooth  curve  on  M  parameterized  by 

:  I  M  where  I  a  subinterval  of  R,  In  local  coordinates,  p  =  C  is 

given  by  k  smooth  functions  . . . ,  whose  derivative  with  respect 

to  t  is  denoted  by  ^{t)  =  We  introduce  an  equivalence  relation 

on  curves  in  M  as  the  basis  of  the  definition  of  tangent  vectors  at  a  point  in  M  [14]. 
Two  curves  and  passing  through  a  point  p  are  said  to  be  equivalent  at  p 
(notation:  ^i(t)  ^  V^2(f))>  if  there  exists  ri,T2  £  I  such  that 

^i(ri)  =  V^2(r2)  =p  (16) 

^i(ri)  =^2(^2).  (17) 

Clearly,  'n/  defines  an  equivalence  relation  on  the  class  of  curves  in  M  passing 
through  p.  Let  [tf)]  be  the  equivalence  class  containing  A  tangent  vector  to  [ip]  is  a 
derivation,  ujp,  such  that  in  local  coordinates  (p^, . . .  ,p^),  it  satisfies  the  condition 
that  given  any  smooth  function  f  \  M  Ry 

v|p(/)(p)  =  Ev-^w^  (18) 

i=o  ™ 

where  p^ip{t).  The  set  of  tangent  vectors  associated  with  all  the  equivalence  classes 
at  p  defines  a  vector  space  called  the  tangent  vector  space  at  p,  denoted  by  TMp, 
The  set  of  tangent  spaces  associated  with  points  in  M  can  be  “glued”  together  to 
form  a  manifold  called  the  tangent  bundle  which  is  denoted  by  TM , 

TM  =  (J  TMp. 

pern 


For  our  purposes,  it  is  important  to  specify  explicitly  how  this  gluing  is  implemented. 
This  will  be  explained  below  after  we  introduce  the  concept  of  a  vector  field  and 
discuss  its  relevance  in  the  model. 

A  vector  field  on  M  is  an  assignment  of  a  derivation  v\p  to  each  point  p  of  M 
which  varies  smoothly  from  point  to  point.  That  is,  if  p  =  (p^,...,p*)  are  local 
coordinates,  then  we  can  always  write  v\p  in  the  form 

3  ^ 

Then  v  is  a  vector  field  if  the  coordinate  functions  A*  are  smooth. 

Comparing  (18)  and  (19)  we  see  that  if  is  a  parameterized  curve  in  M  whose 
tangent  vector  at  any  point  coincides  with  the  value  of  u  at  a  point  p  =  ^(t),  then 
in  the  local  coordinates  p  =  (V^^  (^), . . . ,  V^*(t)),  we  must  have 

^'(t)  =  A^  (p)  for  j  =  1, . . . ,  k.  (20) 

In  our  application,  each  command  issued  by  the  MAHCA  agent  is  implemented 
as  a  vector  field  in  M.  Each  agent  constructs  its  command  field  as  a  combination 
of  ‘primitive’  predefined  vector  fields.  Since  the  chosen  topology  for  M,  0,  is  not 
metrizable,  we  cannot  guarantee  a  unique  solution  to  (20)  in  the  classical  sense 
for  a  given  initial  condition.  However,  they  have  solutions  in  a  class  of  continuous 
trajectories  in  M  called  relaxed  curves  [33].  In  this  class,  the  solutions  to  (20)  are 
unique.  We  discuss  the  basic  characteristics  of  relaxed  curves  as  they  apply  to  our 
process  control  formulation  and  implementation  in  Section  4.  Next,  we  describe  some 
of  their  properties  as  they  relate  to  our  plant  model  and  control  process.  For  this 
objective,  we  need  to  introduce  the  concept  of  fiows  in  Af. 

If  u  is  a  vector  field,  any  parameterized  curve  passing  through  a  point  p  in  M  is 
called  an  integral  curve  associated  with  v  if  in  local  coordinates  (5)  holds.  An  integral 
curve  associated  with  a  field  v,  denoted  by  !?(t,p)  is  termed  the  fiow  generated  by  v 
if  it  satisfies  the  following  properties: 

^{t,  =  ^{t  +  r,p)  (semigroup  property)  (21) 

5?(0,p)=p  (initial  condition) 
and 

^!p(t,p)  =  V  \^(t,p)  (flow  generation) 

Now  we  are  ready  to  customize  these  concepts  for  our  model.  Let  >  0  be 
the  width  of  the  current  decision  interval,  +  A),  Let  C“(p,t)  be  the  unsatisfied 
demand  at  the  beginning  of  the  interval.  Agent  Ai  has  a  set  of  primitive  actions: 

{vij  :  j  =  1, . . . ,  rii  where  Vi^j  \p  e  TMp  for  each  pern}.  (22) 

During  the  interval  [t,t  -h  A)y  agent  Ai  schedules  one  or  more  of  these  actions  to 
produce  a  fiow  which  will  bring  the  system  closer  to  the  goal.  In  particular,  Ai  deter¬ 
mines  the  fraction  aijip^t)  of  A  that  action  Vij  must  be  executed  as  a  function  of 
the  external  perturbation  functions  5r,t(t,  p)  and  the  vector  of  the  behavior  functions 
of  the  agents  in  the  network  B{p,t)  =  (Hi(p,t),...,B;v(t)).  Figure  3  conceptually 


Fig.  3.  Conceptual  illustration  of  agent  action  schedule 


illustrates  a  schedule  of  actions  involving  three  primitives.  We  will  use  this  example 
as  means  for  describing  the  derivation  of  our  model.  The  general  case  is  similar. 

The  flow  l?t  associated  with  the  schedule  of  Figure  3  can  be  computed  from  the 
flows  associated  with  each  of  the  actions: 


!Pj(T,p) 


i'r,p)iit<T  <t  +  Ai,ni 

^■>’1,111  (^>P))  if  ^  "I"  ^i,ni  ^  ^  ^  "I"  ^t,ni  +  ^»,n2 

(t,  (t,  (t,  P))) 

if  t  +  Ai,ni  +  ^»,nj  ^  f 


(23) 


where  A  =  Ai,„j  +Ai,„j  +Ai,„j  and  a<,„,  +a<,„j  +  ai,„3  =  1.  We  note  that  the  flow 
given  by  (23)  characterizes  the  evolution  of  the  process  as  viewed  by  agent  Aj. 
The  vector  fleld  Vi\p  associated  with  the  flow  !Pj  is  obtained  by  differentiation  and 
the  third  identity  in  (21).  This  vector  field  applied  at  p  is  proportional  to 


N.nu 


(24) 


where  is  the  Lie  bracket  due  to  the  parallelogram  law,  see  [42].  The  Lie  bracket 
is  defined  as  follows.  Let  v  and  to,  be  derivations  on  M  and  let  /  :  M  -4  i?  be  any 
real  valued  smooth  function.  The  Lie  bracket  of  v  and  w  is  the  derivation  defined  by 


[v,w]{f)  =  v{w{f))  -  w{vif)), 

see  [10]. 

Thus  the  composite  action  Vijp  generated  by  the  i-th  agent  to  control  the  process 
is  a  composition  of  the  form  of  (24).  Moreover  from  a  version  of  the  Chattering 


lemma  and  duality  [18],  we  can  show  that  this  action  can  be  expressed  as  a  linear 
combination  of  the  primitive  actions  available  to  the  agent  as  follows. 

5l7j(a)  =  l 


where  the  coefficients  jj  determined  by  the  fraction  of  time  that  each  primitive 
action  Vij  is  used  by  agent  i. 

The  effect  of  the  field  defined  by  the  composite  action  Wjjp  on  any  smooth  function 
(equivalent  function  class)  is  computed  by  expanding  the  right  hand  side  of  (23)  in 
a  Lie-Taylor  series  [10].  In  particular,  we  can  express  the  change  in  the  behavior 
modification  functions  (7“  due  to  the  fiow  over  the  interval  A  in  terms  of  The 
evolution  of  the  modified  behavior  function  C7“  over  the  interval  starting  at  point  p 
is  given  by 

^^(f  +  A,p")  =  cnt,  !P<(t  +  A,p))  (26) 

Expanding  the  right  hand  side  of  (26)  in  a  Lie-Taylor  series  aroimd  (t,p),  we 
obtain, 


3 

where 

and 

(t'i|p)'’(/)=/forall/. 


(27) 


In  general,  the  series  in  the  right  hand  side  of  (27)  will  have  coimtable  many  non¬ 
zero  terms.  In  our  case,  since  the  topology  of  M  is  finite  due  to  the  fact  that  it  is 
generated  by  finitely  many  logic  clauses,  this  series  will  have  only  finitely  many  non¬ 
zero  terms.  Intuitively,  this  is  so  because  in  computing  powers  of  derivations  (i.e., 
limits  of  differences),  we  need  only  to  distinguish  among  different  neighboring  points. 
In  our  formulation  of  the  topology  of  M,  this  can  only  be  imposed  by  the  information 
in  the  clauses  of  the  agent’s  Knowledge  Base.  Since  each  agent’s  Knowledge  Base 
has  only  finitely  many  clauses,  there  is  a  term  in  the  expansion  of  the  series  in  which 
the  power  of  the  derivation  goes  to  zero  because  we  cannot  distinguish  between 
two  points  which  cannot  be  separated  by  open  sets  in  the  finite  topology.  This 
is  important  because  it  allows  the  series  in  the  right-hand  side  to  be  effectively 
generated  by  a  locally  finite  automaton.  We  will  expand  on  the  construction  of  this 
automaton  in  the  next  section  when  we  discuss  the  inference  procedure  carried  out 
by  each  agent. 

We  note  that  given  the  set  of  primitive  actions  available  to  each  agent,  the 
composite  action  is  determined  by  the  vector  of  fraction  functions  aj  .  We  will 
see  in  the  next  section  that  this  vector  is  inferred  by  each  agent  from  the  proof  of 
existence  of  solutions  of  an  optimization  problem. 


Now  we  can  write  the  specific  nature  of  the  model  formulated  in  expression  (13). 
At  time  t  and  at  point  pE  M  the  behavior  modification  function  of  agent  i  is  given 
by: 

cr(p,t) = cr(p,t-) + Sr,i(p,  t)+j2  Qi,kBk(p,  n  m 

k 

where  t~  is  the  end  point  of  the  previous  update  interval,  5r,i  is  the  external  per¬ 
turbation  function  to  agent  z,  and  Qi^k  is  a  multiplier  determining  how  much  of  the 
current  behavior  modification  requirements  of  agent  Ak  is  allocated  to  agent  Ai,  This 
allocation  is  determined  from  the  characteristics  of  the  process  both  agents  are  con¬ 
trolling  and  fi:om  the  process  description  encoded  in  the  agent’s  knowledge  base.  The 
actual  request  for  service  from  agent  k  to  agent  i  is  thus  the  term, 

The  information  sent  to  agent  %  by  agent  k  is  the  behavior  modification  function 
at  the  end  of  the  previous  interval.  Finally  the  point  p  E  M  carries  the 
current  status  of  the  process  monitored  by  the  agents  appearing  in  (28).  Agent  k 
thus  contributes  to  Agent  i’s  new  control  only  if  Qi^k  ^  0. 

This  concludes  our  description  of  the  model.  For  space  considerations,  some  de¬ 
tails  have  been  left  out.  In  particular  those  related  to  the  strategy  for  activation  and 
deactivation  of  agents.  These  will  be  discussed  in  a  future  paper. 

4  The  Multiple  Agent  Hybrid  Control  Architecture 

In  this  section,  we  describe  the  main  operational  and  functional  characteristics  of 
our  intelligent  controller.  As  we  mentioned  in  the  introduction,  this  controller  is 
implemented  as  a  distributed  system  composed  of  agents  and  a  communication  net¬ 
work.  We  referred  to  the  latter  as  the  control  network  in  the  previous  section.  The 
architecture  realizing  this  system,  called  the  Multiple-Agent  Hybrid  Control  Ar¬ 
chitecture  (MAHCA),  operates  as  an  on-line  distributed  theorem  prover.  At  any 
update  time,  each  active  agent  generates  actions  as  side  effects  of  proving  an  exis¬ 
tentially  quantified  subtheorem  (lemma)  which  encodes  the  desired  behavior  of  the 
logic  communications  network  as  viewed  by  the  agent.  The  conjunction  of  lemmas 
at  each  instant  of  time,  encodes  the  desired  behavior  of  the  entire  network. 

Each  agent  of  MAHCA  consists  of  five  modules:  a  Planner,  a  Dynamic  Knowledge 
Base,  a  Deductive  Inferencer,  an  Adapter  and  a  Knowledge  Decoder.  We  briefly 
overview  the  functionality  of  an  agent  in  terms  of  its  modules. 

The  Knowledge  Base  stores  the  requirements  of  operations  or  processes  con¬ 
trolled  by  the  agent.  It  also  encodes  system  constraints,  inter-agent  protocols  and 
constraints,  sensory  data,  operational  and  logic  principles  and  a  set  of  primitive 
inference  operations  defined  in  the  domain  of  equational  terms. 

The  Planner  generates  a  statement  representing  the  desired  behavior  of  the  sys¬ 
tem  as  an  existentially  quantified  logic  expression  herein  referred  to  as  the  Behavior 
Statement. 

The  Inferencer  determines  whether  this  statement  is  a  theorem  in  the  theory 
currently  active  in  the  Knowledge  Base.  If  the  behavior  statement  logically  follows 
from  the  current  status  of  the  knowledge  base,  the  inferencer  generates,  as  a  side 
effect  of  proving  this  behavior  statement  to  be  true,  the  current  control  action  sched¬ 
ule.  If  the  behavior  statement  does  not  logically  follow  from  the  current  status  of 


the  knowledge  base,  that  is,  the  desired  behavior  is  not  realizable,  the  inferencer 
transmits  the  failed  terms  to  the  Adapter  module  for  replacement  or  modification. 

Finally,  the  Knowledge  Decoder  translates  data  fi:om  the  other  agents  and  incor¬ 
porates  them  into  the  Knowledge  Base  of  the  agent. 

In  each  agent  of  MAHCA,  the  behavior  statement  is  the  formulation  of  a  re¬ 
laxed  variational  optimization  problem  whose  successful  resolution  produces  an  ac¬ 
tion  schedule  of  the  form  of  (24).  Each  agent  operates  as  a  real-time  theorem  prover 
in  the  domain  of  relaxed  variational  theory  [39].  A  customized  version  of  this  the¬ 
ory,  enriched  with  elements  of  differential  geometry,  equations!  logic  and  automata 
theory  provides  a  general  representation  for  the  dynamics,  constraints,  requirements 
and  logic  of  the  control  network  .  We  devote  the  rest  of  this  section  to  the  discussion 
of  the  main  elements  of  this  theory  in  the  context  of  the  operational  features  of 
MACHA. 

The  Mchitectiure  is  composed  of  two  items:  The  Control  Agent,  and  the  control 
network.  These  items  are  illustrated  in  Figures  4  and  5  respectively.  We  will  discuss 
them  in  the  remaining  of  this  section. 


4.1  Architectural  Elements  of  a  control  agent: 

We  will  discuss  next  the  functionality  of  the  five  modules  of  a  control  agent.  These 
are:  the  Knowledge  Base,  the  Planner,  the  Inferencer,  the  Knowledge  Decoder  and 
the  Adapter. 


4.1.1.  Knowledge  Base:  The  Knowledge  Base  consists  of  a  set  of  equational  first 
order  logic  clauses  with  second  order  extensions.  The  syntax  of  clauses  is  similar  to 
the  ones  in  the  Prolog  language.  Each  clause  is  of  the  form 

Head  ■(-  Body  (29) 

where  Head  is  a  functional  form,  taking  values  in  the  binary  set 

[true,  false]  and  xi,X2,.-.,x„  are  variables  or  parameters  in  the  domain  M  of  the 
MAHCA  network.  The  symbol  stands  for  logical  implication.  The  variables  ap¬ 
pearing  in  the  clause  head  are  assumed  to  be  universally  quantified.  The  Body  of  a 
clause  is  a  conjunction  of  one  or  more  logical  terms 

ei  A  62  A  . . .  A  e„  (30) 

where  A  is  the  logical  ‘and’.  Each  term  in  (30)  is  a  relational  form.  A  relational 
form  is  one  of  the  following:  an  equational  form,  an  inequational  form,  a  covering 
form,  or  a  clause  head.  The  logicsJ  value  of  each  of  these  forms  is  either  true  or 
false.  A  relational  form  Cj  is  true  precisely  at  the  set  of  tuples  of  values  Si  of  the 
Hnmnin  taken  by  the  variables  where  the  relational  form  is  satisfied  and  is  false  for 
the  complement  of  that  set.  Thus  for  e,  =  ai{xi, . .  .,Xn),  Si  is  the  possibly  empty 
subset  of  M” 

Si  =  {(zi , . . . ,  x„)  e  M’* :  ei(xi , . . . ,  x„)  =  true} 
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Fig.  4.  Control  Agent 
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Fig.  5.  Network  of  Cooperating  Control  Agents 


so  that 


e<(xi, . . .  ,Xn)  =  false  if  (xi, . . .  ,x„)  €  M^/So 


The  generic  structure  of  a  relational  form  is  given  in  Table  1. 


Form 

Structure 

Meaning 

equations! 
inequational 
covering 
clause  head 

u;(xi,...,x„)  =  i;(xi,...,x„) 
i«(xi,...,x„)  ^v{xi,...,x„) 
w(xi  9  •  •  •  )  Xfi)  ^  v{x\  )  •  •  •  ;  Xfi) 
q(xi^  •  •  •  fXn) 

equal 
not  equal 
partial  order 
recursion,chaining 

Table  1.  Structure  of  the  Relational  Form 


In  Table  1,  w  and  v  are  polynomic  forms  with  respect  to  a  finite  set  of  operations 
whose  definitional  and  property  axioms  are  included  in  the  Knowledge  Base.  A 
polynomic  form  u  is  an  object  of  the  form  v{xi , . . . ,  Xn)  =  where  f2*  is 

the  free  monoid  generated  by  the  variable  symbols  {xi , . . . ,  Xn}  under  juxtaposition. 
The  term  (u,  u)  is  called  the  coefficient  of  v  at  u.  The  coeSicients  of  a  polynomic 
form  V  take  values  in  the  domain  of  definition  of  the  clauses.  The  domain  in  which 
the  variables  in  a  clause  head  take  values  is  the  manifold  M  described  in  section  2. 
The  logical  interpretation  of  (29)  and  (30)  is  that  the  Head  is  true  if  the  conjunction 
of  the  terms  of  Body  are  jointly  true  for  instances  of  the  variables  in  the  clause  head. 
M  is  contained  in  the  Cartesian  product  : 

MCGxSxXxA  (31) 

where  G  is  the  space  of  goals,  S  is  the  space  of  sensory  data,  X  is  the  space  of 
controller  states  and  A  is  the  space  of  control  actions.  These  were  described  in 
section  2.  G,  5,  X,  and  A  are  manifolds  themselves  whose  topological  structure 
are  defined  by  the  specification  clauses  in  the  Knowledge  Base,  see  Figure  6.  These 
clauses,  whidi  are  application  dependent,  encode  the  requirements  on  the  closed- 
loop  behavior  of  the  model  of  the  agent.  In  fact  the  closed  loop  behavior,  which  we 
will  define  later  in  this  section  in  terms  of  a  variational  formulation,  is  characterized 
by  continuous  curves  with  values  in  M.  This  continuity  condition  is  central  because 
it  is  equivalent  to  requiring  the  system  to  look  for  actions  that  make  the  closed  loop 
behavior  satisfy  the  requirements  of  the  plant  model. 

The  denotational  semantics  of  each  clause  in  the  knowledge  base  is  one  of  the 
following: 

1.  a  conservation  principle, 

2.  an  invariance  principle,  or 

3.  a  constraint  principle. 

Conservation  principles  are  one  or  more  clauses  about  the  balance  of  a  particular 
process  in  the  dynamics  of  the  system  or  the  computational  resources.  For  instance, 
equation  (28)  encoded  as  a  clause  expresses  the  conservation  of  demand  in  the  logic 
communications  network 


conaervationuof  jun8ati8fied-demand{pit\Qi^u]iSr,iy[B}t]iA^C^(tip)) 

/*  encoding  of  equation  (13)  */ 


CJ*(t+/l,p')=C?(t,p)+5..i(t,p)+X)fc  Qi.fcBfc(t,p,p)  A 

/*  encoding  of  equation  (15)  */ 
procc««_ct>o/ti«on(p,t,p")  A  /*  encoding  of  equation  (13)  */ 

can«ct;ot»on_o/_tin«aiw/»e<i_demond(p",i+il,[Qi,fc),Sr,i,[5fc],Zi,C“(<+24,p'^) 


(32) 


In  (32),  the  first  equational  term  relates  the  segment  car  density  for  agent  i  at 
the  current  time  to  the  density  in  the  past  and  the  net  current  density  of  the  other 
agents  connected  to  agent  i.  The  last  term  of  the  rule  implements  the  recursion. 

As  another  example,  consider  the  following  clause  representing  conservation  of 
computational  resources: 


comp{LoadiProcess^Opucount,Limit)  4—  procea8(proce8S^count) 

A  proce88^ount-Loadx—Op^count<Load 
A  Loadi<Limit 

A  comp{Loadi  ,Procea«,Op-cotint,Ifim«t). 

where  Load  corresponds  to  the  current  computational  burden,  measured  in  VIPS 
(Variable  Instantiations  Per  Second),  Process  is  a  clause  considered  for  execution, 
and  Opjcount  is  the  current  number  of  terms  in  process. 

Conservation  principles  always  involve  recursion  whose  scope  is  not  necessarily  a 
single  clause,  as  in  the  example  above,  but  with  chaining  throughout  several  clauses. 

Invariance  principles  are  one  or  more  clauses  establishing  constants  of  the  evo¬ 
lution  of  agent’s  behavior  modification  functions  in  a  general  sense.  These  principles 
include  stationarity,  which  plays  a  pivotal  role  in  the  formulation  of  the  theorems 
proved  by  the  architecture,  and  geodesics.  The  necessary  conditions  for  maximizing 
throughput  in  the  freeway  network  constitute  an  example  of  this  type  of  principle. 
The  importance  of  invariance  principles  lies  in  the  reference  they  provide  for  the 
direction  of  unexpected  events. 

Constraint  principles  are  clauses  representing  engineering  limits  to  actuators  or 
sensors  and,  most  importantly,  behavior  policies.  The  clauses  defining  the  capacity 
function  in  each  segment  are  examples  of  these  type  of  principles.  Another  example  in 
this  application  is  given  by  the  clauses  that  define  the  lifting  strategy  for  embedding 
discrete  varying  trajectories  into  M  (interpolation  rules). 

The  clause  database  is  organized  in  a  nested  hierarchical  structure  illustrated  in 
Figure  6.  The  bottom  of  this  hierarchy  contains  the  equations  that  characterize  the 
algebraic  structure  defining  the  terms  of  relational  forms,  i.e  an  algebraic  variety 
[45]. 

At  the  next  level  of  the  hierarchy,  three  types  of  clauses  are  stored:  Generic 
Control  Specifications,  System  Representation  and  Goal  Class  Representation. 


Laws  of  the  Variety 


Fig.  6.  Knowledge  Base  Organization 


The  Generic  Control  Specifications  are  clauses  expressing  general  desired  behav¬ 
ior  of  the  system.  They  include  statements  about  stability,  complexity  and  robustness 
that  are  generic  to  the  class  of  declarative  rational  controllers.  These  specifications 
are  written  by  constructing  clauses  that  combine  laws  of  the  kind  which  use  the 
Horn  clause  format  described  earlier.  Examples  of  these  types  of  clausse  are  the  one 
that  specifies  the  range  of  the  parameters  in  the  traffic  model  for  which  the  system 
is  stable. 

The  Process  Representation  is  given  by  clauses  characterizing  the  dynamic  behav¬ 
ior  and  structure  of  the  plant,  which  includes  sensors  and  actuators.  These  clauses 
are  written  as  conservation  principles  for  the  d3mamic  behavior  and  as  invariance 
principles  for  the  structure.  As  in  the  case  of  Generic  Control  Specifications,  they  are 
constructed  by  combining  a  variety  of  laws  in  the  equations!  Horn  clause  format.  An 
example  of  this  type  of  clause  are  the  density  and  velocity  equations  of  the  fireeway 
segments  given  in  section  2  encoded  in  Horn  clause  format. 

The  Goal  Class  Representation  contains  clauses  characterizing  sets  of  desirable 
operation  points  in  the  domain  (points  in  the  manifold  M).  These  clauses  are  ex¬ 
pressed  as  soft  constraints;  that  is,  constraints  that  can  be  violated  for  finite  intervals 
of  time.  They  express  the  ultimate  purpose  of  the  controller  but  not  its  behavior 
over  time.. 

The  next  level  of  the  hierarchy  involves  the  Control  Performance  Specifications, 
These  are  typically  problem  dependent  criteria  and  constraints.  They  are  written  in 
equational  Horn  clause  format.  They  include  generic  constraints  such  as  speed  and 
time  of  response,  and  qualitative  properties  of  state  trajectories  [39]. 

Dynamic  Control  Specifications  are  equations!  Horn  clauses  whose  bodies  are 
modified  as  a  function  of  the  sensor  and  goal  commands. 


Finally,  Model  Builder  Realization  clauses  constitute  a  recipe  for  building  a  pro¬ 
cedural  model  (an  automaton,)  for  generating  variable  instantiation  (unification), 
and  for  theorem  proving. 


4.1.2.  The  Planner: 

The  function  of  the  theorem  Planner,  which  is  domain-specific,  is  to  generate, 
for  each  update  interval,  a  symbolic  statement  of  the  desired  behavior  of  the  system, 
as  viewed,  say  by  agent  j,  throughout  the  interval.  The  theorem  statement  that  it 
generates  has  the  following  form. 

Given  a  set  of  primitive  actions  there  exists  a  control  schedule  Vi\p  of  the 
form  (25)  and  a  firaction  function  differential  da(-)  (Figure  3)  in  the  control 
interval  [f,f  -f  such  that  da(-)  minimizes  the  functional 

/•t+a 

Li{9i{T,p),Vi\p{G{i{T,p)))dxxijp,dT)  (33) 

subject  to  the  following  constraints: 


+  =  Gi{t,Xi) 

(local  goal  for  the  interval), 

m 

(inter-agent  constraint,  see  (28)) 


da(p,  dr)  =  1 


In  (33),  Li  is  the  Local  Relaxed  Lagrangian  of  the  system  as  viewed  by  Agent  i 
for  the  current  interval  of  control  [t^t  + A).  This  function,  which  maps  the  Cartesian 
product  of  the  state  and  control  spaces  into  the  real  line  with  the  topology  defined 
by  the  clauses  in  the  knowledge  base,  captures  the  dynamics,  constraints  and  re¬ 
quirements  of  the  system  as  viewed  by  agent  i.  The  relaxed  Lagrangian  function 
Li  is  a  continuous  projection  in  the  topology  defined  by  the  knowledge  base  (see 
[36])  in  the  coordinates  of  the  i-th  agent  of  the  global  Lagrangian  function  L  that 
characterizes  the  system  as  a  whole. 

In  (34),  p  represents  the  state  of  the  process  under  control  as  viewed  by  the  agent 
and  Gi  is  the  parallel  transport  operator  bringing  the  goal  to  the  current  interval, 
see  [27].  The  operator  Gi  is  constructed  by  lifting  to  the  manifold  the  composite  fiow 
(see  equation  (23)).  We  note  that  the  composite  fiow  and  the  action  schedule  are 
determined  once  the  fraction  function  is  known  and  that  this  function  is  the  result 
of  the  optimization  (33),  (34).  In  particular,  the  action  schedule  is  constructed  as  a 
linear  combination  of  primitive  actions,  see  equation  (25). 

The  term  da(-)  in  (33)  is  a  Radon  probability  measure  [40]  on  the  set  of  primitive 
control  actions  or  derivations  that  the  agent  can  execute  for  the  interval  [t,  t-f  ^).  It 
measures,  for  the  interval,  the  percentage  of  time  to  be  spent  in  each  of  the  primitive 


Fig.  7.  Illustration  of  optimization 


derivations.  The  central  function  of  the  control  agent  is  to  determine  this  mixture 
of  actions  for  each  control  interval.  This  function  is  carried  out  by  each  agent  by 
inferring  from  the  current  status  of  the  knowledge  base  whether  a  solution  of  the 
optimization  problem  stated  by  the  current  theorem  exists,  and,  if  so,  to  generate 
corresponding  actions  and  state  updates.  Figure  7  illustrates  the  relations  between 
the  primitive  actions  and  the  fraction  of  A  they  are  active  in  the  interval  [t,  t  +  Zi). 

The  expressions  in  (34)  constitute  the  constraints  imposed  in  the  relaxed  op¬ 
timization  problem  solved  by  the  agent.  The  first  one  is  the  local  goal  constraint 
expressing  the  general  value  of  the  state  at  the  end  of  the  current  interval.  The 
second  represents  the  constraints  imposed  on  the  agent  by  the  other  agents  in  the 
network.  Finally,  the  third  one  indicates  that  is  a  probability  measure.  Under  relax¬ 
ation  and  with  the  appropriate  selection  of  the  domain,  see  [21],  the  optimization 
problem  stated  in  (33)  and  (34)  is  a  convex  optimization  problem.  This  is  important 
because  it  guarantees  that  if  a  solution  exists,  it  is  unique  up  to  probability,  and 
also,  it  guarantees  the  computational  effectiveness  of  the  inference  method  that  the 
agent  uses  for  proving  the  theorem. 

The  construction  of  the  theorem  statement  given  by  (33)  and  (34)  is  the  central 
task  carried  out  in  the  Planner.  It  characterizes  the  desired  behavior  of  the  process 
as  viewed  by  the  agent  in  the  current  interval  so  that  its  requirements  are  satisfied 
and  the  system  “moves”  towards  its  goal  in  an  optimal  manner. 


4.1 .3.  Adapter: 

The  function  under  the  integral  in  (33)  includes  a  term,  referred  to  as  the  “catch¬ 
all”  potential,  which  is  not  associated  with  any  clause  in  the  Knowledge  Base.  Its 


function  is  to  measure  unmodeled  dynamic  events.  This  monitoring  function  is  car¬ 
ried  out  by  the  Adapter  which  implements  a  generic  commutator  principle  similar  to 
the  Lie  bracket  discussed  in  section  3.1,  see  (24).  Under  this  principle,  if  the  value  of 
the  catch-all  potential  is  empty,  the  current  theorem  statement  adequately  models 
the  status  of  the  system.  On  the  other  hand,  if  the  theorem  fsuls,  meaning  that  there 
is  a  mismatch  between  the  current  statement  of  the  theorem  and  system  status, 
the  catch-all  potential  carries  the  equational  terms  of  the  theorem  that  caused  the 
failure.  These  terms  are  negated  and  conjuncted  together  by  the  Inferencer  accord¬ 
ing  to  the  commutation  principle  (which  is  itself  defined  by  equation2il  clauses  in 
the  Knowledge  Base)  and  stored  in  the  Knowledge  Base  as  an  adaptation  dynamic 
clause.  The  Adapter  then  generates  a  potential  symbol,  which  is  characterized  by 
the  adaptation  clause  and  corresponding  tuning  constraints.  This  potential  is  2idded 
to  criterion  for  the  theorem  characterizing  the  interval. 

The  new  potential  symbol  and  tuning  constraints  are  sent  to  the  Planner  which 
generates  a  modified  loc2il  Lagrangian  for  the  agent  and  goal  constraint.  The  new 
theorem,  thus  constructed,  represents  adapted  behavior  of  the  system.  This  is  the 
essence  of  reactive  structural  adaptation  in  the  our  model 

At  this  point,  we  pause  in  our  description  to  address  the  issue  of  robustness. 
To  a  large  extent,  the  adapter  mechanism  of  each  controller  agent  provides  the  sys¬ 
tem  with  a  generic  and  computationally  effective  means  to  recover  from  failures  or 
unpredictable  events.  Theorem  failures  are  symptoms  of  mismatches  between  what 
the  agent  thinks  the  system  looks  like  and  what  it  really  looks  like.  The  adaptation 
clause  incorporates  knowledge  into  the  agent’s  Knowledge  Base  which  represents  a 
recovery  strategy.  The  Inferencer,  discussed  next,  effects  this  strategy  as  part  of  its 
normal  operation. 


4.1.4.  Inferencer: 

The  Inferencer  is  an  on-line  equational  theorem  prover.  The  class  of  theorems  it 
can  prove  are  represented  by  statements  of  the  form  of  (33)  and  (34),  expressed  by 
an  existentially  quantified  conjunction  of  equational  terms  of  the  form: 

3Z (Wi(Z,p)  reli  Vi {Z,p)  A  ...  A  W„(Z,p)  reh  Vn{Z,p))  (35) 

where  Z  is  a  tuple  of  variables  each  taking  values  in  the  domain  Z7,  p  is  a  list  of 
parameters  in  D,  and  {Wi,Vi}  are  polynomial  terms  in  the  semiring  polynomial 
algebra: 

D{n)  (36) 

where  D  =  (D,  (+,  •,  1, 0))  a  semiring  algebra  with  additive  unit  0  and  multiplicative 
unit  1.  In  (35),  reU,  i  =  1, . . . ,  n  are  binary  relations  on  the  polynomial  algebra.  Each 
reli  can  be  either  an  equality  relation  (= ),  inequality  relation  (j4  ),  or  a  partial  order 
relation  (<).  In  a  given  theorem,  more  than  one  partial  order  relation  may  appear. 
In  each  theorem,  at  least  one  of  the  terms  is  a  partial  order  relation  that  defines  a 
complete  lattice  on  the  algebra  that  corresponds  to  the  optimization  problem.  This 
lattice  has  a  minimum  element  if  the  optimization  problem  has  a  minitrinm  Given 
a  theorem  statement  of  the  fonn  of  (35)  and  a  knowledge  base  of  equational  clauses, 
the  Inferencer  determines  whether  the  statement  logically  follows  from  the  clauses 


in  the  Knowledge  Base,  and  if  so,  as  a  side  effect  of  the  proof,  generates  a  non-empty 
subset  of  tuples  with  entries  in  M  giving  values  to  Z.  These  entries  determine  the 
agent’s  actions.  Thus,  a  side  effect  is  instantiation  of  the  agent’s  decision  variables. 
In  (36),  is  a  set  of  primitive  unary  operations,  {u*},  the  infinitesimal  primitive 
fields  defined  in  section  3.  Each  v*  maps  the  semiring  algebra,  whose  members  are 
power  series  involving  the  composition  of  operators  from  Z  to  itself 

Ui :  D{{Z))  ^  D{{Z)).  (37) 

These  operators  are  characterized  by  axioms  in  the  Knowledge  Base  and  are  pro¬ 
cess  dependent.  In  formal  logic,  the  implemented  inference  principle  can  be  stated 
as  follows:  Let  S  be  the  set  of  clauses  in  the  Knowledge  Base.  Let  ^  represent 
implication.  Proving  the  theorem  means  to  show  that  it  logically  follows  from  i7,  i.e. 

S  theorem,  (38) 

The  proof  is  accomplished  by  sequences  of  applications  of  the  following  inference 
axioms: 

(i)  equality  axioms 

(ii)  inequality  axioms 

(iii)  partial  order  axioms 

(iv)  compatibility  axioms 

(v)  convergence  axioms 

(vi)  knowledge  base  axioms 

(vii)  limit  axioms 

The  specifics  of  these  inference  axioms  can  be  found  in  [13]  where  it  is  shown  that 
each  of  the  inference  principles  can  be  expressed  as  an  operator  on  the  Cartesian 
product 

D{{W))  X  D{{W)).  (39) 

Each  inference  operator  transforms  a  relational  term  into  another  relational  term. 
The  inferencer  applies  sequences  of  inference  operators  on  the  equational  terms  of 
the  theorem  until  these  terms  are  reduced  to  either  a  set  of  ground  equations  of  the 
form  of  (40)  or  it  determines  that  no  such  ground  form  exists. 

Zi  =  ai,aieD  (40) 

The  mechanism  by  which  the  inferencer  carries  out  the  procedure  described  above 
is  by  building  a  procedure  for  variable  goal  instantiation  which  in  our  case  is  a  locally 
finite  automaton.  We  refer  to  this  automaton  as  the  Proof  Automaton.  This  impor¬ 
tant  feature  is  unique  to  our  approach.  The  proof  procedure  is  customized  to  the 
particular  theorem  statement  and  Knowledge  Base  instance  it  is  currently  handling. 
The  structure  of  the  proof  automaton  generated  by  the  Inferencer  is  illustrated  in 
Figure  8. 

In  Figure  8,  the  initial  state  represents  the  equations  associated  with  the  theo¬ 
rem.  In  general,  each  state  corresponds  to  a  derived  equational  form  of  the  theorem 


Inference  Rules:  Ai  .  ,  dj  «  e| 

Fig.  8.  Conceptual  Structure  of  the  Proof  Automaton 


through  the  application  of  a  chain  of  inference  operators  to  the  initial  state  that  is 
represented  by  the  path, 

infi  m/2  infk  „ 

00  _ ^  Ol  _ ^  •  •  ■  _ ^  Ok* 

Each  edge  in  the  automaton  corresponds  to  one  of  the  possible  inferences.  A  state  is 
terminal  if  its  equational  form  is  a  tautology,  or  it  corresponds  to  a  canonical  form 
whose  solution  form  is  stored  in  the  Knowledge  Base.  In  traversing  the  automaton 
state  graph,  values  or  expressions  are  assigned  to  the  variables.  In  a  terminal  state, 
the  equational  terms  are  all  ground  states  (see  (40)).  If  the  automaton  contains 
at  least  one  path  starting  in  the  initial  state  and  ending  in  a  terminal  state,  then 
the  theorem  is  true  with  respect  to  the  given  Knowledge  Base  and  the  resulting 
variable  instantiation  is  a  valid  one.  If  this  is  not  the  case,  the  theorem  is  false. 
The  function  of  the  complete  partial  order  term  present  in  the  conjunction  of  each 
theorem  provable  by  the  inferencer  is  to  provide  a  guide  for  constructing  the  proof 
automaton.  This  is  done  by  transforming  the  equational  terms  of  the  theorem  into 
a  canonical  fixed  point  equation,  called  the  Kleene-Schutzenberger  Equation  (KSE) 
[13],  which  constitutes  a  blueprint  for  the  construction  of  the  proof  automaton.  This 
fixed  point  coincides  with  the  solution  of  the  optimization  problem  formulated  in 
(33)  (34),  when  it  has  a  solution.  The  general  form  of  KSE  is 

Z  =  E{p)-Z‘^T{p)  (41) 

In  (41),  E  is  a,  square  matrix,  with  each  entry  a  rational  form  constructed  from 
the  basis  of  inference  operators  described  above,  and  T  is  a  vector  of  equational 
forms  from  the  Knowledge  Base.  Each  non-empty  entry,  Eij^  in  E  corresponds  to 


the  edge  in  the  proof  automaton  connecting  states  i  and  j.  The  binary  operator 
between  E{p)  and  Z  represents  the  "apply  inference  to”  operator.  Terminal  states 
are  determined  by  the  non-empty  terms  of  T.  The  p  terms  are  custom  parameter 
values  in  the  inference  operator  terms  in  £?(*). 

A  summary  of  the  procedure  executed  by  the  inferencer  is  presented  in  Figure  9. 


equality 
inequality 
partial  order 
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convergence 
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KS-equation 
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decompostion 


Fig.  9.  Summary  of  Inferencer  Procedure 


We  note  that  the  construction  of  the  automaton  is  carried  out  from  the  canoni¬ 
cal  equation  and  not  by  a  non-deterministic  application  of  the  inference  rules.  This 
approach  reduces  the  computational  complexity  of  the  canonical  equation  (low  poly- 
nomic)  and  is  far  better  than  applying  the  inference  rules  directly  (exponential). 

The  automaton  is  simulated  to  generate  instances  of  the  state,  action  and  eval¬ 
uation  variables  using  an  automaton  decomposition  procedure  [41]  which  requires 
nlog2n  time,  where  n  is  the  number  of  states  of  the  automaton.  This  “divide  and 
conquer”  procedure  implements  the  recursive  decomposition  of  the  automaton  into 
a  cascade  of  parallel  unitary  (one  initial  and  one  terminal  state)  automata.  Each  of 
the  resulting  automata  on  this  decomposition  is  executed  independently  of  the  oth¬ 
ers.  The  behavior  of  the  resulting  network  of  automata  is  identical  with  the  behavior 
obtained  from  the  original  automaton,  but  with  feasible  time  complexity. 

The  Inferencer  for  each  control  agent  fulfills  two  functions,  (i)  to  generate  a  proof 
for  the  system  behavior  theorem  of  each  agent  generated  by  the  Planner  (equations 
(33)  and  (34))  and  (ii)  to  function  as  the  central  element  in  the  Knowledge  De¬ 
coder.  We  now  describe  its  function  for  proving  the  behavior  theorem.  Later,  we  will 
overview  its  function  as  part  of  the  Knowledge  Decoder.  To  show  how  the  Inferencer 
is  used  to  prove  the  Planner  theorem,  (33),  (34),  first,  we  show  how  this  theorem  is 


transformed  into  a  pattern  of  the  form  of  (35).  Since  (33),  (34)  formulates  a  convex 
optimization  problem,  a  necessary  and  sufficient  condition  for  optimality  is  provided 
by  the  following  dynamic  programming  formulation. 

Vi{Y,  t)  =  in  fat  J  Y),  Vi\p  (G<(r,p)))da(p,  dr)  (42) 

-infaiH{Y, 

where  Y{t)  —  p  and  r  e  [t^t  A) 

In  (42),  the  function  VJ,  called  the  optimal  cost-to-go  function,  characterizes 
minimality  starting  from  any  arbitrary  point  inside  the  current  interval.  The  second 
equation  is  the  corresponding  Hamilton-Jacobi-Bellman  equation  for  the  problem 
stated  in  (33)  and  (34)  where  H  is  the  Hamiltonian  of  the  relaxed  problem.  This 
formulation  provides  the  formal  coupling  between  deductive  theorem  proving  and 
optimal  control  theory.  The  inferencer  allows  the  real-time  optimal  solution  of  the 
formal  control  problem  resulting  in  intelligent  distributed  real-time  control  of  the 
multiple-agent  system.  The  central  idea  for  inferring  a  solution  to  (42)  is  to  expand 
the  cost-to-go  function  V’(., .)  in  a  rational  power  series  V  in  the  algebra 

D{{iY,T))).  (43) 

Replacing  V  for  Vj  in  the  second  equation  in  (42)  gives  two  items:  a  set  of  polynomic 
equations  for  the  coefficients  of  V  and  a  partial  order  expression  for  representing  the 
optimality.  Because  of  convexity  and  rationality  of  V*,  the  number  of  equations  to 
characterize  the  coefficients  of  V  is  finite.  The  resulting  string  of  conjunctions  of 
coefficient  equations  and  the  optimality  partial  order  expression  are  in  the  form  of 
(35).  A  detailed  algorithmic  approach  to  solving  (42)  which  we  call  hybrid  dynamic 
programming  can  be  found  in  [28] 

In  summary,  for  each  agent,  the  inferencer  operates  according  to  the  following 
procedure. 

Step  1:  Load  current  theorem  (33),  (34). 

Step  2:  Transform  theorem  to  equational  form  (35)  via  (42). 

Step  3:  Execute  proof  according  to  figure  9. 

If  the  theorem  logically  follows  from  the  Knowledge  Base  (i.e.,  it  is  true),  the 
inferencer  procedure  will  terminate  on  step  3  with  actions.  If  the  theorem  does  not 
logically  follow  from  the  Knowledge  Base,  the  Adapter  is  activated,  and  the  theorem 
is  modified  by  the  Planner  according  to  the  strategy  outlined  above.  This  mechanism 
is  the  essence  of  reactivity  in  the  agent.  Because  of  relaxation  and  convexity,  this 
mechanism  ensures  that  the  controllable  set  of  the  domain  is  strictly  larger  than  the 
mechanism  without  this  correction  strategy. 


4.1.5  Knowledge  Decoder: 

The  function  of  the  Knowledge  Decoder  is  to  translate  knowledge  data  from  the 
network  into  the  agent’s  Knowledge  Base  by  updating  the  inter-agent  specification 
clauses.  These  clauses  characterize  the  second  constraint  in  (42).  Specifically,  they 


express  the  constraints  imposed  by  the  rest  of  the  network  on  each  agent.  They  also 
characterize  the  global-to-local  transformations  (see  [23]).  Finally,  they  provide  the 
rules  for  building  generalized  multipliers  for  incorporating  the  inter-agent  constraints 
into  a  complete  unconstrained  criterion,  which  is  then  used  to  build  the  cost-to-go 
function  in  the  first  expression  in  (42).  A  generalized  multiplier  is  an  operator  that 
transforms  a  constraint  into  a  potential  term.  This  potential  is  then  incorporated  into 
the  original  Lagrangian  of  the  agent  which  now  accoimts  explicitly  for  the  constraint. 

The  Knowledge  Decoder  has  a  built-in  inferencer  used  to  infer  the  structure  of  the 
multiplier  and  transformations  by  a  procedure  similar  to  the  one  described  for  (14). 
Specifically,  the  multiplier  and  transformations  are  expanded  in  a  rational  power 
series  in  the  algebra  defined  in  (43).  Then  the  necessary  conditions  for  duality  are 
used  to  determine  the  conjunctions  of  equational  forms  and  a  partial  order  expression 
needed  to  construct  a  theorem  of  the  form  of  (42)  whose  proof  generates  a  multiplier 
for  adjoining  the  constraint  to  the  Lagrangian  of  the  agent  as  another  potential. 

The  conjunction  of  equational  forms  for  each  global-to-local  transformation  is 
constructed  by  applying  the  following  invariant  imbedding  principle: 

For  each  agent,  the  actions  at  given  time  t  in  the  current  interval,  as  com¬ 
puted  according  to  (42),  are  the  same  actions  computed  at  t  when  the  for¬ 
mulation  is  expanded  to  include  the  previous,  current,  and  next  intervals. 

By  transitivity  and  convexity  of  the  criterion,  the  principle  can  be  analytically 
extended  to  the  entire  horizon.  The  invariant  imbedding  equation  has  the  same 
structure  as  the  dynamic  programming  equation  given  in  (42),  but  with  the  global 
criterion  and  global  Hamiltonians  instead  of  the  corresponding  local  ones. 

The  local-to-global  transformations  are  obtained  by  inverting  the  global-to-local 
transformations  which,  in  turn,  are  obtained  by  expressing  the  invariant  embedding 
equation  as  an  equational  theorem  of  the  form  of  (35).  These  inverses  exist  because 
of  convexity  of  the  relaxed  Lagrangian  and  the  rationality  of  the  power  series. 

It  is  important  at  this  point  to  interpret  the  functionality  of  the  Knowledge 
Decoder  of  each  agent  in  terms  of  what  it  does.  The  multiplier  described  above 
has  the  effect  of  aggregating  the  rest  of  the  system  and  the  other  agents  into  an 
equivalent  companion  system  and  companion  agent,  respectively,  as  viewed  by  the 
current  agent.  This  is  illustrated  in  Figure  10. 

The  aggregation  model  (Figure  10)  describes  how  each  agent  perceives  the  rest  of 
the  network.  This  unique  feature  allows  us  to  characterize  the  scalability  of  the  archi¬ 
tecture  in  a  unique  manner.  Namely  in  order  to  determine  computational  complexity 
of  an  application,  we  have  only  to  consider  the  agent  with  the  highest  complexity 
(i.e.,  the  local  agent  with  the  most  complex  criterion)  and  its  companion. 


4.2  Architectural  Elements  of  a  Declarative  Control  Network 

The  inter-agent  communication  network’s  main  function  is  to  transfer  inter-agent 
constraints  among  agents  according  to  a  protocol  written  in  equational  Horn  clause 
language.  These  constraints  include  application  dependent  data  and,  most  impor¬ 
tantly,  inter-agent  synchronization.  The  inter-agent  synchronization  strategy  is  very 
simple.  An  agent  is  synchronous  with  respect  to  the  network  if  its  inter-agent  con¬ 
straint  multiplier  is  continuous  with  respect  to  the  current  instance  of  its  active 


System 


Synchronization 


Fig.  10.  The  Companion  Agent 


knowledge.  Since  the  equational  Horn  clause  format  allows  for  the  effective  test  of 
continuity  (which  is  implicitly  carried  out  by  the  inferencer  in  the  Knowledge  De¬ 
coder),  a  failure  in  the  Knowledge  Decoder  theorem  results  in  a  corrective  action 
toward  re-establishing  continuity  with  respect  to  the  topology  defined  by  the  current 
instance  of  the  Knowledge  Base  [20]. 

The  specification  of  the  geometry  of  the  network,  as  a  function  of  time,  is  dictated 
primarily  by  global  observability.  By  global  observability,  we  mean  the  closure  of  the 
knowledge  of  the  system  as  whole  relative  to  the  scope  the  systems  reactivity.  One  of 
the  central  tasks  in  any  application  is  to  provide  knowledge  in  the  equational  clause 
format  to  characterize  global  observability  for  the  hybrid  system. 


4.3  Summary  the  MAHCA  Architecture 


Our  formulation  gives  a  precise  statement  of  a  hybrid  control  problem  in  terms  of  a 
multiple  agent  hybrid  declarative  control.  Our  approach  characterizes  the  problem 
via  a  knowledge  base  of  equational  rules  that  describes  the  dynamics,  constraints 
and  requirements  of  plant. 

For  the  traffic  control  application,  the  Behavior  statement  or  ATC  statement  is 
represented  as  a  Lagrangian  of  the  form 
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where  1?  =  {(x,  f):0<f<oo,0<x<  2.8} 


The  Goal  is  maximize  the  average  throughput,  given  as: 
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The  sensor  data  is  given  by 


(45) 
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where  i  =  1, 2, , . . ,  9  to  represent  sensor  inputs  at  one-third  mile  intervals  and  r  = 
1, 2, ...  seconds. 

Finally,  the  actions  for  the  traffic  control  consist  solely  of  the  commanded  velocity 
vector 

We  have  developed  a  canonical  representation  of  interacting  networks  of  con¬ 
trollers.  Given  a  connectivity  graph  with  N  nodes  (controllers)  and  the  correspond¬ 
ing  agent’s  knowledge  bases,  a  network  of  2N  agents  can  be  constructed  with  the 
same  input-output  characteristics,  so  that  each  agent  interacts  only  with  another 
(equivalent)  companion  agent,  whose  knowledge  base  is  an  abstraction  of  the  knowl¬ 
edge  in  the  network.  Thus,  in  general,  the  multiple-agent  controller  for  any  network 
configuration  is  reduced  to  a  set  of  agent  pairs. 

One  agent  of  the  agent  pair  maintains  coordination  with  other  agent  pairs  across 
the  network.  We  call  that  agent  of  the  pair  which  represents  network  information 
the  Thevenin  Agent,  after  the  author  of  a  similar  theorem  in  electrical  network 
theory.  The  proof  carried  out  by  the  Thevennin  Agent  generates,  as  a  side  effect, 
coordination  rules  that  define  what  and  how  often  to  communicate  with  other  agents. 
These  rules  also  define  what  the  controller  needs  from  the  network  to  maintain 
intelligent  control  of  its  physical  plant. 

Our  approach  develops  a  canonical  way  to  prove  the  theorem  characterizing  the 
desired  behavior  for  each  agent  by  constructing  and  executing  on-line  a  finite  state 
machine  called  the  ’’proof  automaton.”  The  inference  process  is  represented  as  a  re¬ 
cursive  variational  problem  in  which  the  criterion  is  an  integral  of  a  function  called 
the  Generalized  Lagrangian.  The  Generalized  Lagrangian  maps  the  Cartesian  prod¬ 
uct  of  equational  rules  and  inference  principles  to  the  real  line,  thus  effectively  pro¬ 
viding  a  hill-climbing  heuristic  for  the  inference  strategy  of  the  theorem  prover  (see 
section  3),  In  MAHCA  the  inference  steps  play  a  role  analogous  to  action  signals 
in  conventional  control  while  the  vector  fields  on  the  carrier  manifold  M  constitute 
generators  of  feedback  laws  (section  2). 


5  Simulation 

5.1  Simulation  Parameters  and  Operation 

The  simulation  parameters  and  constants  for  the  demonstration  runs  were 

T  Length  of  simulation  run  =  20  minutes,  with  data  snapshots  of  20  second  intervals 
(fixed). 


kz  Percentage  of  cars  complying  with  commanded  velocity  varied  from  5%  to  9%. 

This  rule  was  not  rigid.  The  simulation  runs  demonstrated  adaption  of  this  rule 

(to  a  lower  value)  when  the  system  began  drifting  to  an  instable  condition. 

S  Average  response  time  =  1.5  seconds  (variable). 

C  Car  capacity  coefficient,  determined  for  a  4  lane  freeway. 

Those  parameters  marked  “variable”  were  subject  to  adaptation  by  the  architec¬ 
ture  in  response  to  explicit  or  inferred  changes  in  the  environment. 

Accidents  were  introduced  at  a  random  point  (after  a  100  second  “priming” 
period)  at  a  random  location  between  freeway  position  x  =  1.38  and  x  =  1.43.  At 
this  point  was  reduced  by  one-half. 

The  simulation  uses  a  standard  4-th  order  Runga-Kutta  Integration  scheme  for 
time  and  2-nd  order  Adam-Bashworth  integration  for  space. 

The  simulation  produces  loop  sensor  values  as  inputs  for  the  control  functions. 
Sensors  appear  in  pairs  (120  ft  apart)  at  one-third  mile  intervals  and  provide  velocity 
and  density  data  for  vehicles  and  voids.  The  simulation  simulates  ideal  sensors,  free 
of  bad  data  and  able  to  detect  voids. 

The  simulation  was  implemented  in  Prolog  using  the  same  set  of  equations  as 
those  encoded  into  the  Knowledge  Base.  Although  the  equations  are  the  same,  the 
simulation  is  completely  different  operational  entity  and  only  interacts  with  the 
MAHCA  control  agents  by  providing  them  with  sensor  inputs  and  accepting  their 
control  actions. 

Execution  of  the  simulation  is  performed  by  a  single  top-level  prolog  rule. 


traf  ficjdemo{Input8^  Outputs)  :  — 
simulationjonjof  f  {Stop) , 

8imulation{Inputs,  Sensor  jout^  Action  Jn,  Outputs), 
ccmtroller.on.of  f{Bool),  (46) 

record{Actions-in,  Sensorsjout,  Inputs,  Outputs), 
repeat. 

In  this  rule,  traffic^demo  is  the  main  rule.  The  Inputs  parameter  is  expected  to 
contain  the  operational  parameters  (such  as  maximum  running  time).  The  Outputs 
parameter  will  become  instantiated  with  the  status  of  the  simuluation. 

The  simulation.,on-off  rule  provides  a  mechanism  for  setting  Stop  to  true  to  shut 
down  the  simulation. 

In  Simulation,  the  Inputs  and  Action  Jn  parameters  are  used  as  inputs  to  compute 
the  next  set  of  Sensor^out  values  and  a  new  set  of  simulation  Outputs  (used  for 
recording  purposes  only). 

Like  simulation-on^off,  contoller.on-off  is  a  mechanism  for  setting  Bool  to  True 
to  invoke  the  controller  or  False  to  turn  it  off.  During  the  initial  100  second  priming 
period,  Bool  will  be  False. 

The  controller  rule  invokes  the  inferencing  process  to  compute  a  new  set  of  Ac¬ 
tion-in  values  for  the  simulator  from  the  set  of  Sensor-out  values. 

Finally,  record  will  report,  to  the  terminal  and/or  to  the  data  files,  the  values 
from  the  action  lists,  sensor  outputs,  and  other  simulation  and  controller  inputs  and 


outputs.  The  data  was  recorded  every  60  seconds,  starting  with  t  =  160,  the  end  of 
the  first  control  period  following  the  initial  100  second  priming. 


5.2  Discussion 

The  control  will  compute  and  broadcast  travel  speeds  for  each  segment  of  the  freeway. 
When  followed  by  individuals,  the  throughput  of  the  whole  system  will  increase. 
Compliance  on  the  part  of  drivers  assumes  adoption  of  the  philosophy  that  adherence 
to  less  than  intuitive  speeds  is  in  their  best  interest. 

The  control  can  also  be  used  to  determine  and  transmit  navigational  instructions 
throughout  a  network  of  freeway  corridors,  accomplished  by  tracking  the  wave  prop¬ 
agations  over  the  segments  and  guiding  a  percentage  of  the  vehicles  to  maximize 
throughput  of  the  system  as  opposed  to  minimizing  travel  for  individual  cars. 

In  the  abstraction  of  individual  vehicles  into  a  continuous  wave  model,  some 
information  is  lost.  Since  the  freeway  segment  is  regarded  as  one  unit,  with  a  width 
parameter  representing  the  number  of  lanes,  knowledge  of  any  particular  lane  is 
not  tracked.  Incidences,  real,  such  as  stalled  vehicles,  or  imaginary,  such  as  “rubber 
necking” ,  are  modeled  as  a  reduction  in  the  width  of  the  segment.  Individual  vehicles 
are  simply  contributing  particles  to  the  wave,  so  exact  speed  and  location  of  any  one 
vehicle  is  not  known. 

This  model  can  easily  be  extended  to  perform  other  duties  for  an  intelligent  high¬ 
way  system.  For  example,  detection  of  incidents  can  be  accomplished  by  inspecting 
shock  waves  which  result  for  such  discontinuities.  Prom  the  characteristic  of  the 
shock  wave,  the  nature  of  the  incident  can  be  determined,  e.g.  a  slow  moving  truck 
or  speeding  emergency  vehicle.  This  capability  is  theoretically  possible  and  has  been 
demonstrated  in  a  simulation,  but  it  is  very  difficult  to  validate  with  empirical  data. 
By  comparing  inconsistencies  between  the  observed  and  expected  density  and  veloc¬ 
ity  values  in  the  waves  as  they  propagate  along  the  freeway  corridor,  errant  sensor 
data  can  be  eliminated.  For  traffic  signal  processing,  red  lights  can  be  viewed  as  a 
total  restriction  of  the  highway  and  green  lights  as  an  instantaneous  removal  of  the 
restriction  which  is  naturally  coordinated  with  signal  changes  for  yellow  lights  and 
cross  traffic.  The  modeling  of  voids  becomes  especially  important  in  these  cases. 

The  reactive  capability  of  the  execution  strategy  of  the  architecture  is  exploited 
to  provide  two  major  functions.  Appearance  of  shock  waves  in  the  solution  triggers 
reactivity  to  decompose  the  wave  generator  into  continuous  and  shock  components 
to  reflect  changed  conditions.  Over  the  longer  term,  the  reactivity  can  be  used  to 
fine  tune  the  equations  to  account  for  subtle  changes  in  the  nature  of  the  vehicles, 
such  as  average  length,  and  driver  policies,  such  as  speeds  and  following  distances. 


5.3  Preliminary  Analysis 

This  section  gives  a  brief  preliminary  analysis  of  the  results  of  simulations  that  were 
run  by  Brian  Coles  of  Intermetrics.  The  referenced  charts  appear  at  the  end  of  the 
paper.  There  are  three  sets  of  charts,  one  with  5  %  of  controlled  cars,  one  with  8 
%  of  controlled  cars,and  one  with  9  %  of  controlled  cars.  (With  this  last  set,  the  9 
%  represents  the  initial  percentage  of  controlled  cars.  Over  time,  this  was  lowered 
by  the  Inferencer.)  In  each  set  there  are  three  graphs  showing  the  densities  and 


velocities  of  cars  and  voids,  taken  at  t  =  160,  t  =  700,  and  t  =  1300,  the  end  of  the 
simulation  run.  The  last  two  graphs  in  each  set  show  the  differences  in  velocity  and 
density  values  over  intervals  equal  to  about  1/4  of  the  simulation  run.  For  all  of  these 
graphs,  the  horizontal  units  are  miles  along  the  corridor.  Velocities  are  measured  in 
miles/hour  and  densities  are  measured  in  cars  (or  voids)  per  mile. 

Along  the  portion  of  the  freeway  corridor  before  the  accident,  the  density  wave 
initially  has  a  high  amplitude,  but  as  the  velocity  control  of  the  cars  begins  to 
take  effect,  the  amplitude  of  the  wave  diminishes.  Charts  1-3,  6-8,  and  11-13  show 
this  phenomena  for  samples  of  freeway  dynamics  at  different  times  and  for  different 
percentages  of  control. 

After  the  accident  along  the  freeway,  control  is  not  needed  because  vehicle  den¬ 
sities  never  approach  the  saturation  limit.  Vehicles  are  free  to  travel  as  fast  as  they 
want, within  legal  limits,  of  course. 

For  9%  control,  versus  no  control,  the  throughput  increases  28%.  Less  than  5% 
control  yields  unstable  behavior  and  is  not  effective  in  eliminating  grid-lock.  As 
the  percentage  of  controlled  cars  increases  between  5%  and  9%,  the  grid-lock  is 
monotonicaJly  reduced,  compare  charts  3  and  13.  At  9%  control,  the  grid-lock  is 
successfully  cleared.  Between  9%  and  14%,  the  gridlock  is  successfully  cleared,  but 
throughput  is  reduced.  In  an  effort  to  achieve  the  goal  of  maximizing  throughput, 
the  controller  reduces  the  percentage  of  control  vehicles.  Above  14%  control,  the 
system  experiences  performance  degradation  due  to  overdeterminism. 

The  interaction  between  void  and  car  density  dynamics  shown  in  charts  4,  5,  9, 
10,  14,  and  15  demonstrates  many  density  distributions  that  are  possible,  but  can 
not  be  generated  by  representing  the  dynamics  as  car  densities  alone. 

The  wave  segments  generated  in  the  controller  allow  for  easy  decomposition  be¬ 
tween  continuous  and  shock  elements,  see  for  example  charts  1  and  6.  This  capability 
alone  allows  the  control  to  infer  accidents  and  recompute  the  control  law  accordingly. 
We  observed  a  one-to-one  correlation  (as  expected)  between  theorem  failures  and 
the  appearance  of  the  shock  component. 

In  the  accident  region,  the  four  variables  show  peaks,  see  charts  2,  7,  12.  These 
peaks,  however,  do  not  appear  at  the  same  freeway  locations.  There  are  phase  shifts 
which  are  due  to  the  asymetry  between  the  car  and  void  particles.  The  void  velocities 
has  a  phase  shift  over  car  velocities,  see  charts  4,  5,  9, 10, 14  and  15.  This  phase  shift 
is  expoited  by  the  controller  to  schedule  the  commanded  velocity  profile  to  meet  the 
goal  of  maximizing  throughput.  The  effects  of  the  dynamics  due  to  an  incident  are 
detected  earliest  in  the  void  density  dynamics. 

Void  and  car  densities  together  appear  to  satisfy  the  additive  conservation  law,  as 
to  be  expected  from  models  for  binary  population  dynamics  (void  and  car  particles). 

The  flow  of  velocity  and  density  waves  qualitatively  agrees  with  emperical  ob¬ 
servation. 

Our  runs  show  that  traffic  congestion  latency  decreases  significantly  as  the  per¬ 
centage  of  controlled  cars  increases.  After  incident  removal  with  no  control,  the  car 
density  goes  to  nominal  after  32  minutes.  With  9%  control  the  car  density  goes  to 
nominal  after  only  12  minutes.  This  difference  is  directly  due  to  the  scheduling  of 
velocities  before  the  accident.  That  is,  the  command  velocity  distribution  before  the 
accident  under  control  is  lower  than  the  velocity  distribution  with  no  control 
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